Telephone connections have always been subject to impairments in the form of noise, attenuation, distortion, crosstalk, and echo. Such impairments are particularly common to analog portions of the connection, such as along subscriber loops and within frequency domain multiplexing equipment. Digital transmission alleviates many of these problems but introduces new impairments such as quantization noise or glitches. Even using perfect digital transmission for long-haul connections, a typical telephone connection includes many analog components, such as hybrids, where impairments can occur.
A poor connection or a malfunctioning piece of equipment can produce conditions that a telephone customer will find objectionable or intolerable, so that the connection is deemed to be of very poor quality. When there is a high incidence of such poor connections, customers may complain to the service provider or regulatory authorities, or simply change service providers. Perceived quality of telephone connections is therefore a major factor affecting the reputation and marketability of long distance telephone services.
To guard against poor quality, telephone service providers have developed methods to take objective quality measurements upon a telephone line, a piece of equipment, or even an end-to-end telephone connection. These measurements can help the service provider detect and gauge impairments, pinpoint weak elements and correct deficiencies that degrade user perception of quality.
Many such objective measurements are well known and standardized in the art. Empirically-derived thresholds enable analysts to infer the existence and severity of quality problems by comparing measurements to tables of acceptable values. For example, power levels of test signals and quiet channel noise can be measured electronically. It is well known that a certain range of signal levels must reach a telephone receiver to produce acceptable volume at the earpiece and that the C-weighted noise must be substantially less than the signal level to keep users from experiencing unacceptable noise at the earpiece. The combination of signal and noise measures for a particular connection may be used to determine the likelihood that a user would claim to have difficulty in hearing speech through the phone.
For any one of these objective measurements, the effect of extreme values on user perception of quality is clear, and there are coarse thresholds to define “no effect” and “substantial degradation” conditions. However, for intermediate values and combinations of impairments, there is generally no clear division between values representing acceptable and unacceptable connection quality. Speech clarity and perceived connection quality depend on many variables, including, for example, speech content, talker rhythms, subjective perception of the listener, and users' acclimation to their telephone service. As a result, the correlations between values of objective measures and user perception of connection quality are statistical, representing the combined effects of many different kinds of impairments and variations of sensitivity to them among the population.
Earlier work in this field by the present inventor has created mappings between objective measurements and perceived quality, so that, for example when a given circuit is measured in terms of signal level, noise, distortion, cross-talk, and echo using electronic measurement equipment, the mapping predicts the percentage of conversations that would be reported as being significantly impaired or of poor quality as perceived by an average user population. Such mappings have proven to be a powerful tool for analyzing reported impairments and for gauging acceptable performance of a new line or piece of equipment before deployment.
The mapping was produced by creating or finding telephone circuits having various combinations of measurable characteristics and then having a population of callers conduct test calls through such telephone circuits to subjectively gauge the quality of each call. For each test call, the circuit under analysis was rated on a scale of None-Some-Much for each of the impairments manifested to users of the connection. These impairments included noise, volume, distortion, and echo. Each of the subjectively rated impairments was related to the selected objectively measurable characteristics. The statistics from a large number of such empirical trials with ratings of None-Some-Much for each characteristic may be referred to as Service Attribute Test (SAT) data which characterizes the quality of a communications service.
For each test call described above, each caller also provided an opinion score, which was an overall rating of the circuit quality on a numerical scale. Furthermore, each caller also determined whether the overall effect of the impairments was to render the connection as:
unusable (U; rendering the channel entirely unusable),
difficult ( ); causing enough difficulty to require adaptation by the speaker and listener), irritating (I; disturbing but not requiring adaptation by the speaker and listener), noticeable (N; being minor enough to be ignored), or
unnoticeable (O; having no effect on quality).
The percentage of calls or connections that elicit any one of the first three responses (unusable, difficult, or irritating) is called the P(UDI). The P(UDI) is of particular interest to service providers as a meter of customer satisfaction because it has been shown that overall satisfaction decreases as P(UDI) increases, regardless of average opinion score.
Analysis of empirical data including user reports of impairments and perception quality, together with user reports of impairments obtained in conjunction with objective measurements of connection characteristics, then supported a two step development of a means for predicting user perception of quality from objective measurements. First, a model was produced supporting prediction of P(UDI) and average opinion score as a function of percentages of calls with each of the possible combinations of “none,” “some,” and “much” (N, S, M) conditions reported for each of the impairments. Then, objective measurements were correlated with user reports of impairments to predict the proportion of N, S, M ratings likely to be reported by users as a function of the objective measurements. From these two elements, it was thereafter possible to take measurements of the objective characteristics for connections and translate the set of measures obtained into estimates of likely user perception of quality as revealed by the P(UDI) and an average opinion score.
While traditional circuit-switched telephone networks have been extensively characterized by such an approach, the recent trend toward packet-switched telephony (Internet telephony) has created a need to characterize a telephony channel that is subject to some new impairments. Packet switched networks are generally unsuited to ensuring that transmission delay is fixed or that data packets arrive in sequence, or even at all. This behavior is tolerable for transferring data files and messages, because packets arriving out of order can simply be arranged and lost packets can be retransmitted. However, in packet switched telephony, voice signals are digitized and encoded into a steady stream of discrete packets. Any interruptions or delay variations in transmitting voice data packets, even on the order of milliseconds, can affect the ability to reconstruct the voice signal at the receive end. This problem is further magnified when “codecs” are used to compress or encode the digitized voice signal for more efficient use of data transmission bandwidth. If a sophisticated codec is used and the data stream looses a crucial packet required for reconstructing the voice waveform, the result may be heard as a momentary drop-out, garbled speech, distortion, or a buzzing sound.
Voice over an Internet protocol, sometimes abbreviated as “VoIP”, offers many potential technological and economic benefits. However, large-scale deployment of VoIP is hindered by the confusion over how much VoIP transport will affect the user's perception of voice channel quality. For the companies seeking to deploy and gain revenue from VoIP to compete with so-called “toll-quality” telephony service, there is a large risk that users will find the quality of service unacceptable and will revert to using traditional telephone networks.
There is a need to determine what incidences of packet delay and packet loss are tolerable in packet switched telephony without causing any perceptible degradation in service. The traditional telephone network is an established first communications service that has been well characterized by extensive experiments. The packet switched environment may be viewed as a second communications service with both similarities to and differences from the first communications service.
Therefore, where the quality of a first communications service is well characterized and deemed acceptable, there is a need to establish the requisite performance of a second communication service to be comparable to the first communications service. This is particularly important where the second communication service is subject to additional impairments not applicable to the first communications service and where such additional impairments vary in severity and frequency.
The necessary comparison of the first communications service with the second communications service can be effected according to methods disclosed by the present inventor in U.S. patent application Ser. No. 09/778,186. Given empirical data on the effects of loss frame rates and added delay on user perception of quality, the methods disclosed in the previous application teach the taking into account of multiple added impairments, apportioning the influences of each added impairment toward the composite quality requirements, and devising a useful way of expressing the required performance of a communications service subject to the combination of added impairments.
The data required for effecting such comparisons include, in particular, the results of tests in which human users report their perception of speech distortion in the presence of different levels of packet loss. Because different codecs and packet transmission protocols vary in their behavior and susceptibility to packet loss, application of this approach would involve controlled tests for each of the myriad possible combinations of codecs and packet transmission protocols that might be employed in different packet-switched voice services.
To avoid having to perform a large number of subjective tests, it is desirable to accurately estimate the effects of dropped packets or novel sources of noise in a communications service implemented with a newly developed protocol, without first setting up and conducting subjective tests of quality of voice under that protocol.
It is further desirable to accomplish this as much as possible in a way that can still rely upon established statistics related to impairments common to both first and second communications services, so that knowledge of the first communications service may spare considerable time and effort in determining acceptable performance of the second communication service.